{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/letianzj/QuantResearch/blob/master/notebooks/python.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d6lqkX0A-ihT"
      },
      "source": [
        "Some advanced topics of Python\n",
        "\n",
        "- [Numpy](#numpy)\n",
        "- [Pandas](#pandas)\n",
        "- [Other](#other)\n",
        "- [Reference](#reference)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "UryuAjgDbeFq"
      },
      "outputs": [],
      "source": [
        "%matplotlib inline\n",
        "import numpy as np\n",
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Numpy <a name=\"numpy\"></a>\n",
        "\n",
        "\n",
        "Python integer is not a pure integer. The PyOject_HEAD contains a reference count, the type and size information.\n",
        "\n",
        "![image](https://github.com/letianzj/QuantResearch/blob/master/notebooks/img/cint_vs_pyint.png?raw=1)\n",
        "\n",
        "\n",
        "Similarly, a Python list is more than just a list. The items is a pointer to a block of pointers, each of which in turn points to a full Python object like the Python integer. It is called dynamic-type. In contrast, numpy array is of fixed type where a gorup of homogeneous data are saved in a contiguous segment. \n",
        "\n",
        "\n",
        "![image](https://github.com/letianzj/QuantResearch/blob/master/notebooks/img/array_vs_list.png?raw=1)\n",
        "\n",
        "\n",
        "- [Basic Attributes](#numpy_attributes)\n",
        "- [Indexing and Slicing](#numpy_indexing)\n",
        "- [ufunc](#numpy_ufunc)"
      ],
      "metadata": {
        "id": "VU2GhpOwc5YK"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SGSkXMh7beFr"
      },
      "source": [
        "### Basic Attributes <a name=\"numpy_attributes\"></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "F3i7v9Z8beFr",
        "outputId": "dd120c0e-5fd2-427b-9e5d-5e8d91699696",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1, 2],\n",
              "       [1, 2],\n",
              "       [1, 2]], dtype=int32)"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "x = np.full((3, 2), [1, 2], dtype=np.int32)\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "ZpZhPPg4beFs",
        "outputId": "fd2d5e62-d418-40ca-b942-7cbc8062fac1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(3, 2) 6 4 (8, 4)\n",
            "(27744320, False)\n"
          ]
        }
      ],
      "source": [
        "# 3 rows and 2 columns; size = 2x3 = 6; each int32 item has 4 bytes or 4x8=32bits\n",
        "# Jump 2 items or 2x4=8 bytes to the next row; Jump 1 item or 4 bytes to the next column\n",
        "print(x.shape, x.size, x.itemsize, x.strides)\n",
        "print(x.__array_interface__['data'])         # memory address"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "oHtOYhHjbeFt",
        "outputId": "53d1f47b-4274-4176-add1-8123dd5bee04",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(2, 3) 6 4 (12, 4)\n",
            "(27744320, False)\n",
            "(27744320, False)\n"
          ]
        }
      ],
      "source": [
        "# A reshape does not make a copy; it is a view by altering shape and strides\n",
        "y = x.reshape(2,3)\n",
        "print(y.shape, y.size, y.itemsize, y.strides)\n",
        "print(y.__array_interface__['data'])\n",
        "# So is a transpose\n",
        "print((x.T).__array_interface__['data'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "u-qYNr3rbeFu",
        "outputId": "d222fa5c-0da2-48a0-9ab6-8f94650fd1b0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(6,) 6 4 (4,)\n",
            "(25212672, False)\n"
          ]
        }
      ],
      "source": [
        "# Numpy will force a copy if it becomes non-contiguous.\n",
        "xt_flat = (x.T).reshape(6,)\n",
        "print(xt_flat.shape, xt_flat.size, xt_flat.itemsize, xt_flat.strides)\n",
        "print(xt_flat.__array_interface__['data'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cXyyTnASbeFu"
      },
      "source": [
        "### Indexing and Slicing<a name=\"numpy_indexing\"></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "9FfvIuCjbeFv",
        "outputId": "26f4d0c7-66a3-4612-adb0-ea34ff732732",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "1\n"
          ]
        }
      ],
      "source": [
        "# indexing\n",
        "print(x[0,0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "DB2pfmjqbeFv",
        "outputId": "114decd6-5388-4359-8ceb-c14b35ea93ff",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "27744320 27744320\n",
            "[[10  2]\n",
            " [ 1  2]\n",
            " [ 1  2]]\n",
            "[[10  2]\n",
            " [ 1  2]\n",
            " [ 1  2]]\n"
          ]
        }
      ],
      "source": [
        "# slicing x[start:stop:step] is a non-copy view; by modifying the shape and strides\n",
        "y = x[0:2, 0]\n",
        "print(y.__array_interface__['data'][0], x.__array_interface__['data'][0])   # same memory address\n",
        "y[0] = 10\n",
        "print(x)    # x changes\n",
        "y = np.array([5, 5])  # y points to another array, so x does not change\n",
        "print(x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "id": "6IKzW9X6beFv",
        "outputId": "c79388c0-0676-4dc5-e560-7ed95b5d9797",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([2, 0])"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "# Boolean mask\n",
        "np.sum(x < 2, axis=0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "nIa64N8LbeFw",
        "outputId": "3b080ac7-822e-4415-f247-cdd8f59bf428",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[2 1 1]\n",
            "[[10  2]\n",
            " [ 1  2]\n",
            " [ 1  2]]\n"
          ]
        }
      ],
      "source": [
        "# fancy indexing\n",
        "row = np.array([0, 1, 2])\n",
        "col = np.array([1, 0, 0])\n",
        "y = x[row, col]\n",
        "print(y)     # three elements: (0,1), (1,0), (2,0)\n",
        "# fancy indexing returns a copy\n",
        "y[2] = 5\n",
        "print(x)       # x does not change"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Computation"
      ],
      "metadata": {
        "id": "3vE8A7B8dK1S"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "M1 = np.array([[1,2,3], [4,5,6], [7,8,9]])\n",
        "M2 = np.array([[11,12,13], [14,15,16], [17,18,19]])"
      ],
      "metadata": {
        "id": "b_FMTjgEdJit"
      },
      "execution_count": 38,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "M1[0, :]           # by row"
      ],
      "metadata": {
        "id": "XAmfSfgQdmd2",
        "outputId": "2ed5c845-976c-4461-8142-ca301780d0fa",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 2, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "V1 = np.array([1,2,3])\n",
        "V1.shape"
      ],
      "metadata": {
        "id": "27IvGEkbdrWK",
        "outputId": "37a49645-12d3-4a8c-9d0d-f66aa204df99",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(3,)"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "V2 = V1.reshape([-1,1])\n",
        "V2.shape"
      ],
      "metadata": {
        "id": "brK34wITdwk4",
        "outputId": "522f5f15-f574-45f5-a33d-75315fd82617",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(3, 1)"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "M1 * M2       # element-wise"
      ],
      "metadata": {
        "id": "vqArUSp0d2Xr",
        "outputId": "0f9b7847-b7ba-46b0-e9f4-ad50788f7602",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 11,  24,  39],\n",
              "       [ 56,  75,  96],\n",
              "       [119, 144, 171]])"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "M1 * V1       # V[0] * first col of M, V[1] * second col of M, ..."
      ],
      "metadata": {
        "id": "k3Oy88jPeRky",
        "outputId": "efdbcee2-ae42-4bfd-a172-9e78390b9a44",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1,  4,  9],\n",
              "       [ 4, 10, 18],\n",
              "       [ 7, 16, 27]])"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "V1 * M1       # same as M1 * V1"
      ],
      "metadata": {
        "id": "RewiLJAee-KM",
        "outputId": "9a113708-a7e4-4e3b-89a8-a319ff644de7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1,  4,  9],\n",
              "       [ 4, 10, 18],\n",
              "       [ 7, 16, 27]])"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "M1 * V2       # V[0] * first row of M, V[1] * second row of M, ..."
      ],
      "metadata": {
        "id": "mJvKan1UfDvE",
        "outputId": "e7d44132-b2dc-43c9-9a85-224165e6e0d0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1,  2,  3],\n",
              "       [ 8, 10, 12],\n",
              "       [21, 24, 27]])"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "V2 * M1       # same as M1 * V2"
      ],
      "metadata": {
        "id": "LOc1tNHwfO0a",
        "outputId": "bd040af0-da1f-4341-e76e-a4cd18b41d7b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1,  2,  3],\n",
              "       [ 8, 10, 12],\n",
              "       [21, 24, 27]])"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.dot(M1, V1)         # (3, 3)x(3,) => (3,), as if it was (3,1) except for the shape of result"
      ],
      "metadata": {
        "id": "szOz9b9AfRtE",
        "outputId": "d8569ab3-b820-45d6-9b1a-1d619402e1a0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([14, 32, 50])"
            ]
          },
          "metadata": {},
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.dot(V1, M1)         # (3,)x(3, 3) => (3,), as if it was (3,1) except for the shape of result"
      ],
      "metadata": {
        "id": "n2HNmk-jfeUT",
        "outputId": "e839d104-17be-40f6-f493-84754e1a23e8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([30, 36, 42])"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.dot(M1, V2)         # (3,3)x(3,1) => (3,1)"
      ],
      "metadata": {
        "id": "bwBUjgSuflWv",
        "outputId": "11f8b643-0e04-4520-9ef6-c27fb193d52d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[14],\n",
              "       [32],\n",
              "       [50]])"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.dot(V2, M1)        # (3,1)x(3,3) => error"
      ],
      "metadata": {
        "id": "mJFQLKwtfqtm"
      },
      "execution_count": 51,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "M1 @ M2         # @ == np.dot; except for pd.Dataframe where the result is a pd.Dataframe"
      ],
      "metadata": {
        "id": "285Or1_PfzVx",
        "outputId": "b6aacbbd-d4b9-467d-a28a-fefbbf6043a6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 90,  96, 102],\n",
              "       [216, 231, 246],\n",
              "       [342, 366, 390]])"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "M2 @ M1"
      ],
      "metadata": {
        "id": "LfVuyjTkgBR1",
        "outputId": "d20dfd80-ecd2-4599-bd68-50d6aa5af9ad",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[150, 186, 222],\n",
              "       [186, 231, 276],\n",
              "       [222, 276, 330]])"
            ]
          },
          "metadata": {},
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.dot(M1, M2)"
      ],
      "metadata": {
        "id": "kNns5T5VgC1h",
        "outputId": "1db15653-2e4e-479a-f6b7-d9c6cda88162",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 90,  96, 102],\n",
              "       [216, 231, 246],\n",
              "       [342, 366, 390]])"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "np.dot(M2, M1)"
      ],
      "metadata": {
        "id": "a0i_xZAsgF6i",
        "outputId": "c850f70c-7df6-4ae2-9905-a81ff57628c0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[150, 186, 222],\n",
              "       [186, 231, 276],\n",
              "       [222, 276, 330]])"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "For Dataframe, see https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.div.html"
      ],
      "metadata": {
        "id": "93VO20GngIRG"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bFk9qFIDbeFw"
      },
      "source": [
        "### ufunc<a name=\"numpy_ufunc\"></a>\n",
        "\n",
        "Python loop is interpretive therefore slow. By using NumPy's universal functions (ufuncs), you claim that the loop can be vectorized and pushed into the compiled layer."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "gYkOR1NNbeFw",
        "outputId": "8a13c21c-7d21-4edc-9820-3186fc98add3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "10 loops, best of 5: 85.4 ms per loop\n",
            "1000 loops, best of 5: 382 µs per loop\n"
          ]
        }
      ],
      "source": [
        "big_array = np.random.rand(1_000_000)\n",
        "%timeit sum(big_array)\n",
        "%timeit np.sum(big_array)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "5h7qt-yebeFx",
        "outputId": "0d7927bc-4d44-4f54-f99d-3ea53960be29",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[11.,  3.],\n",
              "       [ 2.,  3.],\n",
              "       [ 2.,  3.]])"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ],
      "source": [
        "# broadcasting\n",
        "y = np.ones((3,1))\n",
        "x+y     # y is broadcasted"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S0Qc4R9sbeFx"
      },
      "source": [
        "ufunc advanced: aggregation\n",
        "\n",
        "![image](https://github.com/letianzj/QuantResearch/blob/master/notebooks/img/map_filter_reduce.jpg?raw=1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "4c6UmtA3beFx",
        "outputId": "335c0b4e-9fa4-4392-c7ad-47eebcc3bcdc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[1, 4, 9, 16, 25]\n",
            "[ 1  4  9 16 25]\n"
          ]
        }
      ],
      "source": [
        "# map\n",
        "def square(x):\n",
        "    return x*x\n",
        "\n",
        "x = [1, 2, 3, 4, 5]\n",
        "print(list(map(square, x)))\n",
        "\n",
        "vfunc = np.vectorize(square)\n",
        "print(vfunc(x))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "bPK2OJGybeFx",
        "outputId": "2ca5aa9e-041f-4210-ade4-b323e2dfc2eb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[2, 4]\n",
            "[2 4]\n"
          ]
        }
      ],
      "source": [
        "# filter\n",
        "def check_even(number):\n",
        "    if number % 2 == 0:\n",
        "          return True  \n",
        "\n",
        "    return False\n",
        "\n",
        "even_numbers_iterator = filter(check_even, x)\n",
        "print(list(even_numbers_iterator))\n",
        "\n",
        "x = np.array(x)\n",
        "mask = x%2 ==0\n",
        "print(x[mask])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "5N9ar7TabeFy",
        "outputId": "6f28b21a-fa3b-4cdd-9dc6-8c94fb5d73d7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "15"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ],
      "source": [
        "# reduce\n",
        "np.add.reduce(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cqkzIPF5beFy"
      },
      "source": [
        "## Pandas <a name=\"pandas\"></a>\n",
        "\n",
        "Pandas Series is an indexed numpy array. Index is immutable and ordered (but not necessarily sorted).\n",
        "\n",
        "DataFrame consists of multiple column-oriented series that share an index. Each column has a dtype.\n",
        "\n",
        "- [Grouping](#pandas_grouping)\n",
        "- [Apply](#pandas_apply)\n",
        "- [Merge](#pandas_merge)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "6k5s5WuzbeFy",
        "outputId": "e885dbbc-68dd-40a6-e0e5-d636e8daa041",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                      P   S   Sym\n",
              "2020-01-01 09:30:00.000000002-05:00  10  94  AAPL\n",
              "2020-01-01 09:30:00.000000003-05:00  91  43  AAPL\n",
              "2020-01-01 09:30:00.000000004-05:00  63  31  AAPL\n",
              "2020-01-01 09:30:00.000000005-05:00  20  70  AMZN\n",
              "2020-01-01 09:30:00.000000006-05:00   9  60  AAPL"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-fdc6b694-d252-4765-b40c-77825d5fd091\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>P</th>\n",
              "      <th>S</th>\n",
              "      <th>Sym</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000002-05:00</th>\n",
              "      <td>10</td>\n",
              "      <td>94</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000003-05:00</th>\n",
              "      <td>91</td>\n",
              "      <td>43</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000004-05:00</th>\n",
              "      <td>63</td>\n",
              "      <td>31</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000005-05:00</th>\n",
              "      <td>20</td>\n",
              "      <td>70</td>\n",
              "      <td>AMZN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000006-05:00</th>\n",
              "      <td>9</td>\n",
              "      <td>60</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fdc6b694-d252-4765-b40c-77825d5fd091')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-fdc6b694-d252-4765-b40c-77825d5fd091 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-fdc6b694-d252-4765-b40c-77825d5fd091');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "np.random.seed(0)\n",
        "df_orders = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=['BS', 'B', 'O', 'OS'])\n",
        "df_orders['Sym'] = np.random.choice(['AAPL', 'AMZN'], 100, replace=True)\n",
        "df_orders.index = pd.date_range(\"2020-01-01 09:30:00\", periods=100, freq='ns', tz='US/Eastern')\n",
        "\n",
        "df_trades = pd.DataFrame(np.random.randint(0,100,size=(100, 2)), columns=['P', 'S'])\n",
        "df_trades['Sym'] = np.random.choice(['AAPL', 'AMZN'], 100, replace=True)\n",
        "df_trades.index = pd.date_range(\"2020-01-01 09:30:00\", periods=100, freq='ns', tz='US/Eastern') + pd.Timedelta(2, 'N')\n",
        "\n",
        "df_trades.head(5)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "ve3j36fqbeFy",
        "outputId": "f9e712d6-f05d-414a-f787-5c3bb7ce413d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                      P   S   Sym\n",
              "2020-01-01 09:30:00.000000003-05:00  91  43  AAPL\n",
              "2020-01-01 09:30:00.000000004-05:00  63  31  AAPL\n",
              "2020-01-01 09:30:00.000000005-05:00  20  70  AMZN\n",
              "2020-01-01 09:30:00.000000006-05:00   9  60  AAPL"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-7e1c4b6b-a935-4388-8852-9576ab1d08d0\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>P</th>\n",
              "      <th>S</th>\n",
              "      <th>Sym</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000003-05:00</th>\n",
              "      <td>91</td>\n",
              "      <td>43</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000004-05:00</th>\n",
              "      <td>63</td>\n",
              "      <td>31</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000005-05:00</th>\n",
              "      <td>20</td>\n",
              "      <td>70</td>\n",
              "      <td>AMZN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000006-05:00</th>\n",
              "      <td>9</td>\n",
              "      <td>60</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7e1c4b6b-a935-4388-8852-9576ab1d08d0')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-7e1c4b6b-a935-4388-8852-9576ab1d08d0 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-7e1c4b6b-a935-4388-8852-9576ab1d08d0');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "t1 = df_trades.index[0] + pd.Timedelta(1, 'N')\n",
        "t2 = t1 + pd.Timedelta(3, 'N')\n",
        "df_trades[t1:t2]       # slicing by index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "MPrPeQ6zbeFz",
        "outputId": "a56df39d-d4f7-4ce5-ae2f-05907f155953",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 112
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                      P   S   Sym\n",
              "2020-01-01 09:30:00.000000003-05:00  91  43  AAPL\n",
              "2020-01-01 09:30:00.000000004-05:00  63  31  AAPL"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-80fb719a-28a8-4249-bd44-34816c4928f9\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>P</th>\n",
              "      <th>S</th>\n",
              "      <th>Sym</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000003-05:00</th>\n",
              "      <td>91</td>\n",
              "      <td>43</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000004-05:00</th>\n",
              "      <td>63</td>\n",
              "      <td>31</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-80fb719a-28a8-4249-bd44-34816c4928f9')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-80fb719a-28a8-4249-bd44-34816c4928f9 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-80fb719a-28a8-4249-bd44-34816c4928f9');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ],
      "source": [
        "df_trades[1:3]         # slicing by row number"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "id": "7Si4GmUVbeFz",
        "outputId": "9d89ab26-35f3-422e-cfb6-47133a44336f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 175
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                      P   S   Sym\n",
              "2020-01-01 09:30:00.000000014-05:00  39  24  AMZN\n",
              "2020-01-01 09:30:00.000000034-05:00   7  21  AMZN\n",
              "2020-01-01 09:30:00.000000094-05:00  77  21  AAPL\n",
              "2020-01-01 09:30:00.000000097-05:00  25  21  AAPL"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-475204f4-deb9-48db-be4d-94f6f0766ef6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>P</th>\n",
              "      <th>S</th>\n",
              "      <th>Sym</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000014-05:00</th>\n",
              "      <td>39</td>\n",
              "      <td>24</td>\n",
              "      <td>AMZN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000034-05:00</th>\n",
              "      <td>7</td>\n",
              "      <td>21</td>\n",
              "      <td>AMZN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000094-05:00</th>\n",
              "      <td>77</td>\n",
              "      <td>21</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2020-01-01 09:30:00.000000097-05:00</th>\n",
              "      <td>25</td>\n",
              "      <td>21</td>\n",
              "      <td>AAPL</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-475204f4-deb9-48db-be4d-94f6f0766ef6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-475204f4-deb9-48db-be4d-94f6f0766ef6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-475204f4-deb9-48db-be4d-94f6f0766ef6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ],
      "source": [
        "df_trades[(df_trades.S > 20) & (df_trades.S < 25)]     # masking"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EnbyJ5HwbeFz"
      },
      "source": [
        "### Grouping\n",
        "\n",
        "groupby: split/apply/combine"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZeX-AEZ2beFz"
      },
      "source": [
        "## Reference <a name=\"reference\"></a>\n",
        "\n",
        "* [Numpy QuickStart](https://numpy.org/doc/stable/user/quickstart.html)\n",
        "\n",
        "* [Pandas User Guide](https://pandas.pydata.org/docs/user_guide/index.html)\n",
        "\n",
        "* [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6yfHxG7_beFz"
      },
      "outputs": [],
      "source": [
        ""
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "Untitled2.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}